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Abstract 

High reliability of mission- and safety- critical software 
systems has been identified by NASA as a high-priority 
technology challenge. We present an approach for the per- 
formance analysis of a neural network (NN) in an ad- 
vanced adaptive control system. This problem is impor- 
tant in the context of safety- critical applications that re- 
quire certification , such as flight software in aircraft. We 
have developed a tool to measure the performance of the 
NN during operation by calculating a confidence inter- 
val (error bar) around the NN’s output. Our tool can be 
used during pre- deployment verification as well as mon- 
itoring the network performance during operation. The 
tool has been implemented in Simulink and simulation re- 
sults on a F- 15 aircraft are presented. 


1. Introduction 

The benefits of neural network based systems are 
clear — a system that can rapidly adapt to a changing 
environment can solve many of the complex problems 
we face today. These systems are attracting increas- 
ing attention in applications where autonomy is an im- 
portant feature or where it is virtually impossible to 
analyze, in advance, all possible environmental condi- 
tions that may arise during the mission. Control ap- 
plications which need to react to catastrophic changes 
(e.g., a broken control surface of an aircraft) can also 
benefit from a neural network based system. The appli- 
cation of neural networks within sub disciplines of engi- 
neering is increasing but new methods are required to 
ensure the reliable deployment and operation of these 
systems. In a safety-critical application, no neural net- 
work based system will be used unless verification and 
validation (V&V) can demonstrate its reliability in a 
cost effective manner. The adaptive nature of neural 
networks requires a significantly different approach to 
V&V methods. In order to perform all required V&V 
activities, the software development process (e.g.. as 


prescribed by IEEE standards) need to be adapted 
accordingly (see e.g., [l]). Nevertheless, V&V of neu- 
ral network based control systems is very difficult, be- 
cause the underlying theory of machine learning usu- 
ally does not provide enough mathematically rigorous 
help for verification. Therefore, verification and valida- 
tion must be augmented by specifically tailored valida- 
tion and dynamic monitoring tools (see [4, 5]). 

In this abstract, we focus on a method and tool to 
analyze the performance of a neural network during 
V&V and during operation (as a monitoring device). 
We will illustrate our tool with an adaptive flight con- 
trol system (FCS), as developed within the IFCS (In- 
telligent Flight Control System) project at NASA Dry- 
den. The FCS (Fig. 1) is a straight-forward dynamic 
inverse controller: the pilot stick commands axe mixed 
with the current sensor readings (airspeed, angle of at- 
tack, and altitude) to form the desired behavior of the 
aircraft. The dynamic inverse model then calculates the 
required actuator (e.g., aileron) movements to bring 
the aircraft into the desired state. In nominal opera- 
tion, the entire control is established that way. If, how- 
ever, the aerodynamics of the aircraft changes (e.g., due 
to a broken surface), there is a deviation between de- 
sired and actual state. The neural network is trained 
during operation to produce a correction signal Uad 
to minimize this deviation. The controller and the net- 
work with a Eli architecture are described in detail in 
[3], 



Figure 1. IFCS Adaptive Control Architecture 





2. Confidence Measure and Confidence 
Tool 

For control applications, it is important that the 
controller’s model of the plant reflects the actual sys- 
tem with sufficient accuracy - something which has to 
be demonstrated during V&V of a neural networks 
based system. This means that it needs to be shown 
that for the entire operation envelope, the modeling 
error does not exceed a given threshold. Thus, to mea- 
sure the reliability of the neural network output is of 
great interest. It is usually desirable that a statisti- 
cal error bar is placed on the output of a neural net- 
work which predicts the probability how close the esti- 
mated and the actual values are. The error bar is a pre- 
diction of the range of the model output where the ac- 
tual value lies with a probability of more than 95%. 
With the assumption of a normal distribution of the 
errors, confidence intervals can be related to the stan- 
dard deviation of the network output. 

Using a Bayesian approach, the error bars can be 
calculated, given the history of training data and the 
parameters of the network (e.g., the weights). We have 
developed the tool CT/OLNN to dynamically calcu- 
late the error bars of the network outputs Uad for the 
controller shown in Fig. 1. It is implemented on top of 
the IFCS Simulink model for a F-15. The tool has been 
implemented to handle Eli and single hidden layer neu- 
ral networks. The tool can be modified easily to accom- 
modate other network and control architectures for fu- 
ture applications. 

3. Simulation Results 

Figure 2 shows a graphical representation of the 
tool’s output for three simulations runs for three dif- 
ferent operating conditions. In each graph, the output 
of the NN over time is shown as a solid line. At time 
t = 1.5s, the pilot issues a command. In the nominal 
case (A), the neural network produces some small out- 
put to accommodate for modeling inaccuracies. During 
the time of adaptation, the confidence decreases but 
improves again over the time as neural network con- 
verges. In our figure, we show the error bars (variance) 
as dashed lines. Within this band, the actual (true) 
value is located with a probability of 95%. A broad 
band corresponds to a low confidence value. Figures 2B 
and C depict scenarios when a failure occurs (a con- 
trol surface got stuck at different angles at t = 1.5s). 
In these cases, the NN’s output needs to be substan- 
tially larger to counteract the damage. Also, the width 
of the error band is much larger, indicating a low confi- 
dence in the results of the network. The damage in Fig- 
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Figure 2. Output of the on-line adaptive ANN 
(solid); dashed lines define confidence interval 


ure 2C was found to be already very close to the sta- 
bility limit; the large waves in the output and the very 
low confidence is a clear indication of that. 

4. Conclusions 

The enormous potential of adaptive controllers for 
safety-critical applications is obvious; for example, po- 
tentially major catastrophic failures and loss of life can 
be avoided. Work on development of the NN-adaptive 
technology itself and on its V&V methods is crucial for 
NASA missions as well as beneficial to numerous other 
agencies and the aerospace industry. 

Our CT / OLNN analysis tool provides assistance to 
establish reliability of a neural network based adaptive 
control system. Our technique will enhance the under- 
standing of performance of the adaptive systems and 
address the detection and possible prevention of catas- 
trophic failures. However, for proper V&V of such sys- 
tem, a multitude of technologies and tools are required. 
The development of such tools and processes will help 
to successfully infuse the neural network adaptive tech- 
nology into real missions. 

References 

[1] D. Mackall, S. Nelson, and J. Schumann. Verification 
and Validation of Neural Networks of Aerospace Appli- 
cations. Technical Report CR-211409, NASA, 2002. 

[2] A. Mili, B. Cukic, Y. Liu, and R. Ayed. Towards the Ver- 
ification and Validation of online adaptive Systems. In 
Comp. Methods in SW Eng. Kluwer, 2003. 

[3] R. Rysdyk and A. Calise. Fault tolerant Flight Control via 
adaptive Neural Network augmentation. AIAA, AIAA- 
98-4483:1722-1728, 1998. 

[4] J. Schumann and P. Gupta. On V&V of Neural Network 
based Controllers. In EANN 03, pp 40-47, 2003. 

[5] J. Schumann and S. Nelson. Toward V&V of Neural Net- 
work based Controllers. In WOSS, pp 67-72. ACM, 2002. 

[6] S. Yerramalla, E. Fuller, M. Mladenovski, and B. Cu- 
kic. Lyapunov analysis of Neural Network Stability in an 
adaptive Flight Control System. In Proc. DSN’03, 2003. 


